Skip to content

Compression libraries: FastLZ, LZ4 and Snappy #5820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 20 commits into
base: master
Choose a base branch
from
Draft

Compression libraries: FastLZ, LZ4 and Snappy #5820

wants to merge 20 commits into from
+1,340 −1

Conversation

Amxx
Copy link
Collaborator

@Amxx Amxx commented Jul 28, 2025 •
edited
Loading

Addresses #5817

Done:

  • FastLZ
  • Snappy
  • LZ4

Todo:

  • LibZip breaks lint. Do we want to ignore it somehow (how?) of run lint:fix on it ?
    • Use solady's npm package as a dev-dependency for its JS folder
  • Lorem ipsum breaks codespell. What do we do about that.

PR Checklist

  • Tests
  • Documentation
  • Changeset entry (run npx changeset add)

@changeset-bot changeset-bot
Copy link

changeset-bot bot commented Jul 28, 2025 •
edited
Loading

⚠️ No Changeset found

Latest commit: cd20e50

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@Amxx Amxx force-pushed the feature/compression branch from 30a0a2d to a206a50 Compare July 28, 2025 19:19
@Amxx Amxx changed the title Compression library [WIP - DO NOT REVIEW - SEE ISSUE] Compression library Jul 29, 2025
@socket-security Socket Security
Copy link

socket-security bot commented Jul 30, 2025 •
edited
Loading

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security		 Vulnerability Quality Maintenance License
Addednpm/​lz4js@​0.2.01001008176100
Addednpm/​snappy@​7.3.0971009984100
Addednpm/​solady@​0.1.2410010010095100

View full report

@Amxx Amxx changed the title [WIP - DO NOT REVIEW - SEE ISSUE] Compression library Compression libraries: FastLZ and Snappy Jul 30, 2025
Amxx
Amxx commented Jul 30, 2025
View reviewed changes
Amxx
Amxx commented Jul 30, 2025
View reviewed changes
Amxx
Amxx commented Jul 30, 2025
View reviewed changes
@Amxx Amxx changed the title Compression libraries: FastLZ and Snappy Compression libraries: FastLZ, LZ4 and Snappy Jul 31, 2025
@Amxx
Copy link
Collaborator Author

Amxx commented Jul 31, 2025 •
edited
Loading

Current gas comparaison (update 5b39ebf)

Test lowest gas cost extra cost (Fastlz) extra cost (Snappy) extra cost (LZ4)
0 22228 +0.00% +1.60% +3.62%
1 22555 +0.00% +1.38% +4.07%
2 22579 +0.00% +1.38% +4.02%
3 22591 +0.00% +1.38% +4.01%
4 24830 +0.00% +4.33% +7.02%
5 40980 +0.00% +8.24% +6.73%
6 163724 +0.00% +12.47% +4.33%
7 4543657 +0.00% +13.79% +3.62%
8 4543889 +0.00% +13.81% +3.62%
9 23405 +0.00% +1.72% +5.70%
10 23677 +0.00% +2.50% +6.92%
11 23813 +0.00% +1.47% +6.37%
12 24015 +0.00% +2.53% +7.57%
13 269620 +0.00% +2.34% +0.44%
14 185750 +2.50% +0.00% +0.06%

Note:

  • FastLZ is the simplest, and most efficient decompression implementation
  • FastLZ is the least standard (LZ4 is the more widelly available)
  • Case 0 et 12 are artifficially compressible. We should pmrobably comme up with more realistic blocks of data (historical blobs from L2s?)
  • Maybe we should cosider the compressed size as well (not just the decompression cost)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Amxx